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ABSTRACT 

The Thinking Mathematics project, a joint effort of 
the American Federation of Teachers and the Learning Research and 
Development Center, had the primary objective of developing more 
efficient means of disseminating new knowledge about mathematics 
learning and instruction* The resulting practitioner-researcher 
collaboration developed an instructional approach based on current 
research findings interpreted by the clinical wisdom of classroom 
teachers* This approach, called Thinking Mathematics, formed the 
basis of an inservice training program implemented throughout the 
AFT*s Eaucational Research & Dissemination network* Teacher 
change — and teacher conviction in the changes being advocated — was 
viewed to be critical for successful student learning outcomes* Pilot 
implementation of Thinking Mathematics occurred at five different 
sites across the country during the 1990-91 academic year, involving 
about 65 classes from grades K-5* Cognitive and affective effects 
upon students involved in the year-^long implementation of Thinking 
Mathematics were assessed by us.,ag multiple sources of data: teacher 
self-reports of student change, standardized achievement test scores, 
student attitude survey findings, and problem solving test results. 
The teacher self-report data show that the teachers involved in the 
program perceived empowering changes in their students* Project 
students performed as well as or better than their non-project peers 
on both the Computational and Concepts and App.licationr subsections 
of standardized achievement tests* Student problem solving abilities 
improved, as measured by the Wood-Cobb problem solving tests* 
Positive student attitudes towards mathematics were revealed by the 
project^s attitude survey: studerts* relatively lower motivation 
scale scores, however, suggest that they might not yet be ready to 
match their efforts with their generally positive regard for 
mathematics* Although the results reported should be considered 
preliminary, there are multiple indications that student learning and 
attitudes were enhanced by their participation in the Thinking 
Mathematics program* (Contains- 43 references*) (Author) 
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Abstract 

The Thinking Mathematics project, a joint effort of the American 
Federation of Teachers and the Learning Research and Development 
Center, had the primary objective of developing more efficient means of 
disseminating new knowledge about mathematics learning and instruction. 
The resulting practitioner-researcher collaboration developed an 
instructional approach based on current research findings interpreted by 
the clinical wisdom of classroom teachers. This approach, called Thinking 
Mathematics, formed the basis of an inservice training program 
implemented throughout the AFT's Educational Research & Dissemination 
network. Teacher change ~ and teacher conviction in the changes being 
advocated — was viewed to be critical for successful student learning 
outcomes. Pilot implementation of Thinking Mathematics occurred at five 
different sites across the country during the 1990-91 academic year, 
involving about 65 classes from grades K-5. 

Cognitive and affective effects upon students involved in the year-long 
implementation of Thinking Mathematics were assessed by using muhiple 
sources of data: teacher self-reports of student change, standardized 
achievement test scores, student attitide survey findings, and problem 
solving test results. The teacher self-report data show that the teachers 
involved in the program perceived empowering changes in their students. 
Project students performed as well as or better than their non-project 
peers on both the Computational and Concepts and Applications 
subsections of standardized achievement tests. Student problem solving 
abilities improved, as measured by the Wood-Cobb problem solving tests. 
Positive student attitudes towards mathematics were revealed by the 
project's attitude survey; students' relatively lower motivation scale scores, 
however, suggest that they might not yet be ready to match their efforts 
with their generally positive regard for mathematics. Although the results 
reported should be considered preliminary, there are multiple indications 
that student learning and attitudes were enhanced by their participation in 
the Thinking Mathematics program. 
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Thinking Mathematics: What's in it for the Students? 

Does empowering teachers empower students, and if so, how? There 
is widespread agreement among educational reformers that the answer 
ought to be "Yes" (BUlups & Rauth, 1992; Liebermaii, 1986; Lieberman & 
Miller, 1984). Most reformers also agree that implementing the broad 
changes in mathematics education that are being called for in numerous 
national reports (e.g.. National Board for Professional Teaching Standards, 
1989; National Council of Teachers of Mathematics, 1989, 1991; National 
Research Council, 1989; Mathematical Science Education Board, 1991) is 
both absolutely necessary and extremely difficult. 

The purpose of this paper is to report preliminary findings about the 
extent of the benefits to students when teachers did engage in one such 
reform effort, the Thinking Mathematics (TM) program. Thinking 
Mathematics is a collaborative project between the American Federation of 
Teachers (AFT) and the Leaming Research and Development Center (LRDC) 
at the University of Pittsburgh, The goal of the project was to develop 
more effective ways of disseminating new knowledge about mathematics 
instruction and learning. A group of teachers in collaboration with 
researchers synthesized material to be used in the AFT's Educational 
Research and Dissemination network. ^ This inservice training is designed 
to assist teachers in changing their instructional practice based on the 
research findings (i.e., implementation of the Thinking Mathematics 
program).^ 

The present paper examines the ways in which this particular 
teacher-researcher collaboration impacted upon the teachers and students. 
Data to address these issues were collected from multiple perspectives and 
sources. First, we set a context by discussing some of the issues related to 

1 The ER&D network, begun in 1981, provides a mechanism by which research studies are 
synthesized into Janguage meaningful to teachers and used in professional development 
sessions. The process is designed to bring research findings to classroom teachers. Details 
about this program can be obtained from Lovely Billups, Director of Field Services, 
American Federation of Teachers, 555 New Jersey Avenue NW, Washington, DC 20001. 
^ A brief description of the Thinking Mathematics project is provided in Appendix A. 
More extensive discussions are provided in Leinhardt and Grover (1990) and Bickel and 
Hattrup (1991)- 
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making teachers key figures in the change process, and how this particular 
collaboration addressed those issues. We conclude with a summary of the 
affective and cognitive effects on students in the Thinking Mathematics 
program. 

Teacher Change as a Context for Student Change 

Numerous reasons have been cited in explaining the difficulty of 
making lasting change (Cohen, 1987; Cuban, 1990; Passow, 1986). 
Teachers have been asked to commit to radical reform programs when 
they do not have* sufficient training either in the mathematics or in the 
program. Cohen and Ball frame the issue of concern as follows: "How can 
teachers teach a mathematics that they never learned, in ways that they 
never experienced? That is the dilemma that such reforms pose" (1990, p. 
353). Teachers are asked to commit to reform without sufficient support 
from systems that, in fact, are set up to punish risk-taking behaviors more 
often than to reward them (e.g., Silberman, 1970; Smith, 1991). In 
addition, there exists a lack of teacher conviction about the validity of the 
reform program (e.g., McLaughlin, 1990), which is exacerbated by the lack 
of training and support. Approaches designed to create change without 
addressing these insufficiencies cannot empower teachers. 

Since the mid- 1980s, there has been an increased awareness of the 
need for the professional involvement of teachers in the decision making 
that influences their working conditions. Romberg (1986), in reviewing 
what had been learned from previous experiments in mathematics 
curriculum change, advised that the key to successful mathematics 
innovation is for practitioners to be full partners in developing those 
changes. Teachers axe called upon to take control of their own profession, 
gain power to influence the educational environment, and become a force 
in the restructuring of schools (Shanker, 1985). 

Several programs in which teachers play a major role in successful 

change efforts are described in Lieberman's 1986 book. Rethinking school 

improvement: Research, craft, and concept. For example, the Interactive 
Research and Development on Schooling (IR&DS) program (Jacullo-Noto, 
1986) involved teachers and researchers in a collaborative research and 
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development process that empowered teachers by "provid[ing] recognition, 
reinforcement, and respect for these professionals" (p. 188). The Scarsdale 
Teachers Institute (STI) offers staff development activities that have been 
developed by and are often conducted by teachers, with the support and 
involvement of the Scarsdale Teachers Association, the administration, the 
board of education, and the community. "The autonomy teachers feel 
through the institute structure and their freedom to create professional 
programs provide the impetus for their support. Teachers manage their 
own growth. They run the courses, hire the consultants, manage the 
budgets" (Schwartz, 1986, p. 204). In the mid 1980s, both the state of 
Connecticut and the Stanford University's Teacher Assessment Project 
(TAP) implemented collaborative approaches to developing alternative 
assessments for teachers involving public school teachers, university 
educators, educational researchers, and state department of education staff 
members (Grover, 1991). Vermont, Maine, Massachusetts, and California 
have recently initiated collaborative efforts in developing student 
assessments (Stenmark, 1991). 

These efforts at empowerment, however, are not the norm. 
Educational reformers have generally failed to address the critical issue of 
teacher belief in the validity of the reform programs. Teachers have not 
been integrally involved in either the research forming the basis for 
reform mandates or in the design and implementation of the reform 
program itself. Staff development materials are usually designed and staff 
development activities are usually conducted by teacher educators, in 
consuhation with educational researchers. Contact between teachers and 
researchers has all too often been limited to teachers' preservice education 
or the occasional inservice program run by outside "experts." Not 
surprisingly, this lack of substantive interaction with researchers often 
leads to confusion among teachers about the plethora of reform-minded 
programs, doubts about the feasibility of classroom implementation ("...but 
that couldn't work in nnx school..."), and indifference. Teachers see reforms 
come and go (Cohen, 1987; Cuban, 1990). How can the proponents of 
reform answer these valid concerns of astute practitioners? 
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Education oriented researchers can address teacher concerns in at 
least two ways. One entails fostering and entering into more intensive 
practitioner-researcher collaborations. Work in the area of research 
utilization has shown that both the intensity and duration of researcher- 
practitioner interactions greatly affect the probability of application of a 
piece of research by practitioners (cf. Huberman, 1990). A second is to 
provide teachers with evidence of a program's effectiveness in actual 
classrooms with actual students. A program may work wonders in theory, 
but how many teachers have theoretical classrooms? Both of these 
approaches promote a common goal, increased teacher conviction about 
and participation in education reform. Unfortunately, little is known about 
the influence of the level of teacher empowerment in the implementation 
of innovative mathematics programs. The work of Carpenter, Fennema, 
Peterson, Chiang, and Loef (1989), Cobb et al. (1991), Lane (1991), and 
Stein, Grover, and Silver (1991) represent recent movement in this 
direction. 

The Thinking Mathematics project attempted to increase teacher 
conviction, by engaging in intense practitioner-researcher collaboration, 
and by providing relevant information on student learning outcomes as a 
result of implementing the protect. Empowerment of both teachers and 
students was hypothesized to result because teachers were involved in all 
phases of development and in dissemination^, and attempts were made to 
assess the impact of the project on the students involved. 

The model for practitioner-researcher collaboration that has evolved 
out of the Thinking Mathematics project has itself been the focus of much 
study and reflection (Bickel & Hattrup, 1991/1992; Hattrup & Bickel, 1992; 
Leinhardt & Grover, 1990). In addition, the collaboration's effects upon 
the various communities involved — mathematics education researchers, 
the teachers, and the leaders of the professional teacher organizations— 



^Products of the collaboration ( Thinkin g Mathematics: Vol I Foundations and Thinking 
Mathematics: Vol. 2 Extensions ^ used in dissemination are described briefly in the 
Appendix and can be obtained from the American Federation of Teachers by writing to 
Dale Boatright, American Federation of Teachers, 555 New Jersey Ave. NW, Washington, 
DC 20001. 
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have been examined. Representatives from each community have 
identified important consequences resulting from their participation in the 
Thinking Mathematics project. Researchers are expanding their ideas 
about how to plan, conduct, and interpret research (Resnick, 1992). 
Teachers feel revitalized, are developing networks among their colleagues, 
and are developing an appreciation for their own ability to interpret 
research for instructional practice (Bickel & Hattnip, 1991/1992; Gill & 
Murakami, 1992; Grover, Gill, & Kaduce, 1991). In planning the second 
decade of operating its Educational Research and Dissemination (ER&D) 
program, the leadership of the AFT is taking into account the critical 
importance of teacher-researcher dialogue and the nature of the support 
structures necessary to effectively put research into practice (Billups, 
1992). 

While these are encouraging signs and might be taken to establish 
the potential of the collaborative model developed by the Thinking 
Mathematics project, the picture is incomplete without an assessment of 
the project's impact upon students. Students' increased mathematical 
understanding, ability, and enjoyment are the ultimate aims of any 
mathematical education reform effort. If, in the end, despite radical and 
promising changes observed in the principal collaborators' communities, 
there are no signs of progress toward meeting the project's goals for 
students, careful rethinking of the instructional program must occur. As 
Morgan aptly put it: "Children are the future. Everything we do is for them 
and everything that will be done will be done by them." (1988, cited in 
National Research Council's Everybody Counts . 1989, p. 96). How have the 
"real live" students who have been involved in the Thinking Mathematics 
project's first year of dissemination been affected by this reform-spirited 
effort? What has been in it for the students? 
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The 1990-91 Thinking Mathematics Student Assessment^ 

We were interested in learning about both affective and cognitive 
effects on students of the Thinking Mathematics (TM) implementation. To 
this end, we collected student data using three different types of 
assessments, each of which provided different information about the skills 
and interests of the students. One was a survey of student attitudes 
towards mathematics, another was standardized achievement test score 
data, and the third assessment instrument was a problem solving test. In 
addition, we obtained data from pilot site teachers about their perceptions 
of the influence their participation and their students' participation in the 
Thinking Mathematics project had upon their students. Results from each 
of these data sources will be described and discussed in turn. 

It is important to note that these assessments were not designed to, 
and therefore do not, constitute a formal program evaluation of Thinking 
Mathematics, and they cannot be considered as such. The goal of the 
student assessment was rather to provide timely and relevant preliminary 
information about Thinking Mathematics' impact on students, by 
triangulating from multiple data sources. Each individual assessment 
provides only one perspective, and there are limitations associated with 
each. Taken together, however, they provide us with a richer 
understanding of how Thinking Mathematics impacted upon students and 
teachers on the road to empowerment. 

Results and Discussion 

Data was collected from students and teachers at five sites across the 
country (San Francisco, CA, Albuquerque, NM, Anderson, IN, Gary, IN, and 
Hammond, IN). There were approximately 65 classes, from grades K-5 
(plus one 7th/8th LD class), in which teachers were voluntarily 
implementing the Thinking Mathematics approach during the 1990-91 
school year. Since participation in the project's assessments was also 
voluntary, there are varying numbers of teachers and students whose data 



^See Hojnacki and Grover (1991) for a detailed account of the results of the assessments 
addressed in this section of the paper. 
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contributed to each of the four assessments. Details concerning each 
assessment are provided in the following subsections. 

1. Teachers' Views of Student Progress with Thinking Mathematic s 

Two teacher self-report instruments, a midyear survey and an end- 
of-year evaluation, were developed in order to obtain TM teachers' 
perspectives on the project's impact on themselves and on their students. 
These instruments are self reports of teachers' experiences with Thinking 
Mathematics in their own classrooms. While we do not have independent 
observations to compare to the self-reported observations, the information 
gleaned from the teachers reveals common themes in their subjective 
impressions. Tlie teachers' perceptions of the changes that they made in 
their instructional practice set a context for their perceptions of changes in 
students, and are reported here for that purpose. 

A. Midyear survev . A six-item survey was distributed to all pilot 
site teachers in December 1990, toward the end of their first semester of 
training and classroom implementation of TM (Gill, 1990). Highlighted 
here are some of the views expressed by the 64 pilot teachers in that 
survey. What was quite clear was that the teachers reported their 
instructional practices to be changing. For example, a majority of pilot 
teachers surveyed reported that Thinking Mathematics had caused them to 
make adjustments to their teaching timelines (90%), had changed their use 
of textbooks (with some teachers dropping them completely) (80%), and 
had altered their grading and/or assessment practices (65%). In response 
to the question of "What pleases you most, at this point, about TM?", a 
majority of the responses (59%) were related to student factors: enhanced 
learning, increased motivation, and greater enjoyment in doing 
mathematics. For example, one 2nd grade teacher wrote: "It's exciting to 
see the "light bulb" go on. It takes a long time to teach, but once they have 
the basics, ThirJcing Math is so powerful. The students feel successful and 
powerful." The other 40% of the responses mentioned factors such as 
specific aspects of the TM approach (12%), aspects of TM supporting 
teacher professional development (10%) and collegial sharing (10%), or 
combinations of the above factors (8%). 



Hojnacki & Grovcr 



April 1992 



Thinking Mathematics and Students 



9 



B. End~of~year evaluations. Another evaluation instniment was 
administered to the pilot teachers toward the end of their second semester 
of involvement in TM (May 1991), in order to gain relevant follow-up 
information on teachers' views of Thinking Mathematics program and the 
dissemination process. Teachers received these end-of-year evaluations 
and responded tc them a week prior to a final group "debriefing" meeting 
of the TM teachers at each of the pilot sites. These end-of-year evaluation 
sessions were videotaped at three sites and audiotaped at a fourth site. No 
data on the debriefing session was available from the fifth site, since it was 
neither audiotaped or videotaped. Approximately 15 teachers who 
participated in the group sessions did not individually fill out evaluation 
forms. However, 49 completed evaluation forms were received from ail 
five sites. 

Pilot teachers noted both direct and indirect benefits to students 
through their participation in Thinking Mathematics. Direct affective 
benefits mentioned were improved student self-confidence and self- 
esteem (e.g., through the encouragement to produce -- and accept — 
multiple solutions to problems), and that students learned to value and 
respect each other's opinions. There were also direct cognitive benefits to 
the students perceived by the teachers, such as deeper student 
understanding of mathematics and greater number sense. 

TM teachers viewed their own professional development and 
improved instruction/pedagogy as constituting indirect benefits to their 
students. A common theme expressed in the evaluation sessions was that 
their own enthusiasm and confidence about mathematics was conveyed to 
their students. Also, they found themselves spending more time on 
mathematics instruction by integrating math throughout the curriculum. 
These are positive indications of teacher empowerment via Thinking 
Mathematics. 

But the teachers also expressed some frustrations about their year- 
long implementation of Thinking Mathematics. One was their feeling of not 
having enough time time to plan, time to work through concepts as they 
would have liked to, time to prepare for substitute teachers while they 
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were having TM group meetings (one day a month of release time after the 
initial five days of training), and so forth. They were also concerned with 
"covering the curriculum" by the end of the year, which created a conflict 
with the goal of allowing students sufficient time to develop deep 
understanding of mathematical concepts. Finally, many pilot site teachers 
worried about whether implementing Thinking Mathematics would place 
their students at a disadvantage relative to other students, in terms of 
their performance on standardized tests, which are timed. 

This concern about test scores is by no means unique to the Thinking 
Mathematics project; the press for scores is felt in many other contexts 
nationwide (cf. Mitchell, 1992; Smith, 1991). To study if this concern had 
any basis, we collected standardized achievement test score data on the 
students involved in Thinking Mathematics in 1990-91. The data were 
from those national or state tests already being administered at the pilot 
sites. We also collected two other kinds of student data: (1) student 
attitudes toward mathematics (as measured by responses to the project's 
Attitude Survey); and (2) student problem solving abilities (as measured 
by performance on the Wood-Cobb Problem Solving Tests, Grades One, 
Two, and Three). Resuhs and discussion of our analyses of these three 
data sources follows. 

2. Student Standardized Achievement Test Performance 

Although an increasing number of teachers hold concepts of 
educational attainment that they believe are not adequately captured by 
state-mandated achievement tests, they still feel extreme pressure from 
district administrators and the general public to raise scores. The project's 
primary goal in looking at standardized achievement test scores was to 
address concerns that student scores might be negatively affected by 
exposure to the Thinking Mathematics (TM) approach. To this end, we 
collected standardized test scores from a total of 1713 students in 87 
classes, in Grades K-5. Fifty-one of these classes were pilot testing the 
Thinking Mathematics approach; 36 classes were not. Three different 
standardized tests (i.e., California Achievement Test - CAT, the Indiana 
State Test of Educational Progress - ISTEP, and the Comprehensive Test of 
Basic Skills - CTBS) were administered to students in TM schools. 
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The non-project classes cannot be considered as control groups in the 
usual research sense. Since teachers had voluntarily participated in 
Thinking Mathematics training and classroom implementation, there was 
not random assignment of classes to TM or non-TM conditions. No 
attempts were made to control the instructional formats of teachers in 
non-TM classes to be sure that those teachers did not employ any of the 
TM principles, and no attempts were made to standardize the degree of 
implementation of TM in project classrooms (in keeping with the voluntary 
philosophy of ER&D, of which TM was a part). Because of this lack of true 
control conditions for evaluating the impact of TM on students, formal tests 
of statistical significance and definitive cause/effect conclusions are 
inappropriate. However, some preliminary conclusions are suggested by 
the available data. 

Figure 1 displays the mean test scores (as measured by median 
national percentiles [MDNPs]) for Grades 1 through 5 on the Computational 
subtests for students in TM classes compared to students not in TM classes. 
Except for Grade 1, the differences in .v, iputational scores between TM 
and non-TM classes are relatively small. The non-TM mean computational 
score is higher than the TM mean for Grades 2 and 3, but the reverse is 
observed for Grades 4 and 5. The difference between the TM and non-TM 
classes' computational means is much greater f r Grade 1, however, with 
the TM mean being 25% greater than the non-TM mean. 



Insert Figure 1 about here 



The mean test scores for the Concepts and Applications subtests for 
Grades 1 through 5 are shown in Figure 2. In all grades, the TM classes' 
mean scores are equal to (for Grade 2) or higher than (for Grades 1, 3, 4 
and 5) the non-TM classes' mean scores (again, as measured by MDNPs). 



Insert Figure 2 about here 



Thus, the standardized test data enables us to conclude that TM 
students are doing at least as well or better than their non-TM peers. 
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These data are consistent with the results reported by Schifter and Simon 
(1991) when they compared performance on standardized tests of 
students prior to and following their enrollment in classes of teachers who 
had participated in the Educational Leaders in Mathematics (ELM) project. 
The teachers in ELM implemented a constructivist-oriented approach to 
teaching, an approach consistent with the NCTM standards. No significant 
differences were found in the standardized test scores of approximately 
300 elementary and 300 secondary students involved in the study. 
Schifter and Simon (1991) report the following: 

Yet standardized test scores did not change. This result should 
help allay concerns that greater attention to understanding and 
problem solving, particularly considering the additional time 
allotted to conceptual exploration, will lead to a decline in 
computational skill. The related concern that instructional 
changes of this magnitude will result in lower test scores for 
the first year or two, as teachers leam the ropes, has also been 
expressed. However, these test results indicate that even 
during the initial change process, computational skill is not 
necessarily sacrificed, (p. 48) 

Using student data collected from multiple choice tests as the only 
measure of student mathematical achievement is universally decried as 
being inadequate and misleading (cf. Mitchell, 1992; NCTM, 1989). Our 
purpose in collecting these data was to contribute to a broad sketch of the 
performance of TM students, to be used to supplement the data from the 
project's attitude survey and the problem solving tests. The current 
standardized test data cannot be used to establish that the TM approach 
enhances or diminishes student performance on these standardized tests. 
From this information, it appears that the performance is at least 
comparable to or better than that of students in classes which were not 
part of the pilot implementation. These results, as with Schifter and 
Simon's (1991), should help allay concerns about declines in computational 
skill. 
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3. Student Attitudes towards Mathematics 

The affective aspects of student participation in Thinking 
Mathematics were directly assessed by the project-wide administration of 
an attitude inventory. The "Students' Attitudes Toward Mathematics" 
Survey consisted of 24 items from three scales: a) a Mathematical 
Confidence scale, reflecting how confident students felt about their 
mathematical abilities; b) a Mathematical Motivation scale, reflecting how 
motivated students were to do math (in a classroom setting); and c) a 
General Attitudes towards Mathematics scale, which reflected students' 
more general attitudes about the subject and its learnability. Each scale 
consisted of 8 items. Half of the survey items were phrased positively 
(e.g., Item #1: "I am sure that I can learn math."), and half were phrased 
negatively (e.g.. Item #4: "I forget most of the things that I learn in math.") 
Most of these items were drawn from several existing attitude scales 
(Fennema-Sherman Mathematics Attitudes Scales; Second International 
Mathematics Study; International Study of Achievement in Mathematics; 
Mathematics Attitude Inventory). Additional items were constructed and 
added to the set to equalize the number of positively and negativeiy 
phrased items in each scale. There were also two open-ended questions on 
the survey: a) "What do you like most about math class?", and b) "What 
bothers you the most about math class?". Pilot testing of the Attitude 
Survey items and format was conducted in September 1990. On the basis 
of this pilot testing, we restricted the administration of the survey to those 
students in 2nd grade or above. Kindergarten and 1st grade students did 
not have sufficient understanding of the concepts being addressed in the 
inventory, making the validity and reliability of their responses 
questionable. 

Approximately 540 Thinking Mathematics students participated in 
the attitude survey administered in May 1991. There were three different 
test formats (a 24-item Yes/No format, a 24-item Yes/Maybe/No format, 
and a 22-item format Yes/Maybe/No format). The "maybe" response was 
included as a choice on two of the test formats in order to see how student 
response patterns changed when there was a "maybe" option. The 22-item 
format was an earlier version of the 24-item Attitude Survey and was 
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administered at one of the TM pilot sites. The two 24-item versions were 
identical except for inclusion of the "maybe" response. Preliminary 
analyses of all three test formats indicated that the response patterns on 
the Yes/Maybe/No format surveys were not markedly different from the 
response patterns of the Yes/No format survey. Therefore, detailed 
analyses were conducted only upon the Yes/No format of the survey 
(n=250). The scores reported reflect the percentages of students giving 
responses that indicated positive attitudes about math, regardless of how 
the item was worded. Throughout this discussion, we use the phrase 
"positive responses" to mean responses that indicate a positive attitude 
toward mathematics, not necessarily a "Yes" response. 

Figure 3 displays the mean percentages of positive responses for 
each scale, as well as breakdowns by grade for each scale. Overall, the 
mean percentage of positive responses across all scales and grades was 
79%, indicating that students did indeed have very favorable attitudes 
towards mathematics. Students had the most positive responses to the 
General Attitudes toward Mathematics (General to Mathematics) scale 
items and the least positive responses to the Motivation scale items. Grade 
trends indicate that the 4th graders typically held less positive attitudes 
towards mathematics than the younger grade students did. 



Insert Figure 3 about here 



The item with the highest percentage of positive responses (97%) 
was from the Confidence scale. Item #1: I am sure that I can learn math. 
More than 90% of the students gave a "No" response to two other items, 
from the General to Mathematics scale (Item #14: / dislike everything 
about math, and Item #15: Only a few people can learn math), indicating a 
highly positive attitude towards mathematics. Illustrative of the more 
ambivalent ratings students accorded to Motivation items overall, two of 
the three lowest items were from the Motivation scale (Item #10: / like 
easy math problems the best - 57% "No"; and Item #20: Sometimes I 
work more math problerrs than are assigned in class -- 53% "Yes"). The 
third item representing a somewhat negative attitude was from the 
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Confidence scale (Item #16: / often think "I can't do it" when a math 
problem seems hard - 44% "Yes"). 

One possible explanation for these results5 may be that the longer 
the exposure to negative experiences, the more difficuh it is to change 
attitude. Fourth graders may have had three or four years (K to 3rd 
grade) of negative experiences whereas second graders have had only one 
or two years (K and 1st grade). One year of the TM approach apparently 
has less impact on the more ingrained attitudes of the older students. 

Interestingly, our findings concerning the mathematical attitudes of 
young elementary school children prefigure the attitudes of the 8th and 
I2th grade students taking part in the SIMS study (McKnight et al., 1987). 
McKnight et al. (1987) characterized their older students as perceiving 
themselves as capable in mathematics but with a limited commitment to it. 
Less than 50% of their 8th grade students believed that mathematics was 
enjoyable or spent a lot of their own time doing mathematics. However, 
these students viewed mathematics overall as a largely fixed, rule- 
oriented body of knowledge to be acquired through memorizing and 
learning to apply rules. The Thinking Mathematics approach encourages 
students to develop a very different conception of mathematics, one that is 
much less dogmatic and that values students' own solution strategies as 
much as, if not more than, the traditional algorithms. Ahhough we did not 
similarly explore TM students beliefs about the nature of mathematics, we 
predict that they would not share these views of their SIMS' peers. 

Student responses to the two open-ended questions on the survey 
largely confirmed findings from the Yes/No survey items. The single most 
common response to the question "What do you like most about math 
class?" was that math is fun. The most frequent response to the question 
"What bothers you most about math class?" was "Nothing bothers me about 
math class". 

The available data on student attitudes are limited in a number of 
important ways. We do not have any comparative data, either from the 

5 This explanation was suggested by a third grade teacher-colleague who is active in TM 
dissemination and implementation efforts. 
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Students' attitudes about mathematics prior to their experiences with 
Thinking Mathematics or from the TM students' peers in non~TM 
classrooms. Ahhough it is certainly a positive finding that students enjoy 
mathematics, we cannot conclude that positive affect necessarily translates 
into improved skills and abilities. Indeed, several studies that have 
examined mathematics attitudes/skills relationships have discovered little 
covariation (cf. McKnight et al., 1987; Stevenson, Lummis, Lee, & Stigler, 
1990). Compared to their Asian counterparts, for example, U.S. students 
have better attitudes but less ability (the latter as measured on 
international achievement tests). U.S. students and their parents believe 
that the (U.S.) students are good at math, although international 
comparisons point to other conclusions (McKnight et al., 1987; Stevenson et 
al., 1990). 

4. Student Problem Solving Performance 

In addition to collecting information on students' attitudes and 
students' mathematical achievement (as measured by standardized 
multiple choice tests), we wanted to obtain information about Thinking 
Mathematics students' problem solving abilities, since problem solving is a 
primary instructional focus of the TM approach (see Appendix A). Three 
problem solving instruments for Grades One, Two, and Three designed by 
Wood and Cobb (1989)^ were administered to Thinking Mathematics 
students to assess their problem solving performance prior to and 
following a year of TM instruction. We were interested not only in the 
change in their pretest to posttest accuracy, but also in the kinds of errors 
that students made. Both accuracy and error information have potential 
for diagnostic use in the classroom. 

The 25 -item Grade One test was administered to students in ten first 
grade classes. The 33-item Grade Two test was administered to students 
in ten second grade classes. The 33-item Grade Three test was 



^ In developing these three problem-solving tests. Wood and Cobb's goals were to 
"develop paper-and-pencil items that assess children's understanding [of arithmetic and 
that] offer a useful alternative to the traditional *skills' approach to mathematics 
assessment*' (p. 13). All the items are open-ended. Collectively, they assess children's 
understanding of number concepts, place value, and the four arithmetical operations. 
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administered to students in eleven third grade classes, nine fourth grade 
classes, and one 7th/8th grade class for learning disabled students. Data 
were obtained from 252 students in a random sampling design, 
representing approximately 25% of each class. Pretests were administered 
in September 1990, and posttests in May 1991. More students took the 
posttest than the pretest, since some pilot site teachers did not administer 
pretests. Data from those students for whom both pretest and posttest 
scores were available (N=158) was used for accuracy analyses and 
pre/posttest comparisons. Data from all 252 students, however, entered 
into error analyses. 

A. Accuracy, Common Errors, and Idiosyncratic Errors . Answers to 
the problem solving items were categorized as either being correct 
answers or errors; the 5032 errors were further categorized as being 
either common or idiosyncratic ones. For each item (pretest and posttest), 
specific errors produced by three or more students were noted; these were 
categorized as being common errors, and were the focus of subsequent 
analysis. Our rationale for this focus was that these were predictable 
errors that teachers were most likely to see again. Errors that were made 
by only one or two students were categorized as idiosyncratic ones. [We 
encouraged teachers to explore these types of errors more fully with the 
individual students who made them, but no further analysis was 
performed on them collectively.] 

The mean percentages of responses in each of three categories are 
shown in Figure 4 and Table 1.^ Posttest accuracy rates were much higher 
than pretest rates for all three tests on paired one-tailed t-tests (Gr. 1: 
t(37)=+11.37; Gr. 2: t(44)=+19.65; Gr. 3: t(74)=+l 1.12, all three at £ < .0001), 
with gains ranging from 25% (on the Grade Three test) to 45% (on the 
Grades One and Two tests). Insofar as could be measured by these 
particular tests developed by Wood and Cobb (1989), Thinking 
Mathematics students were definitely learning problem solving skills. As 
is the case for the attitude survey and standardized achievement test 



7 We provided teachers with more detailed information for both accuracy and error 
patterns, to aid them in instructional planning; this data is reported in Hojnacki and 
Grover (1991). 
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scores, however, we did not have suitable comparisons, Thv. items on this 
test were not standardized or norm-referenced. 



Insert 


Figure 


4 about 


here 




Insert 


Table 


1 about 


here 



As can be seen in Table 1, errors on the Grade Three test were split 
about equally, both on the pretest and the posttest, between those that 
were common (Pre: 28%; Post: 14%) and those that were idiosyncratic (Pre: 
27%; Post: 14%). A different pattern emerged for errors on the Grade One 
test. The percentage of errors was split about equally for the pretest, but 
idiosyncratic errors accounted for nearly twice as many errors as common 
errors did on the posttest. The reverse was true on the Grade Two test. On 
the Grade Two test, the percentage of errors was split about equally for the 
posttest . but on the pretest , there were twice as many idiosyncratic as 
common errors. At this point of our analyses, we cannot offer a compelling 
explanation for these patterns. 

We examined pre- and posttest differences in the range of answers - 
- the spread from the smallest incorrect answer to the largest one -- 
produced on the three tests. This provides a measure of the extent of 
random guessing. For example, on Item #1 of the Grade One pretest, the 
answers ranged from "1" to "13" (the correct answer was "10"), a range of 
12. The mean range of answers on each of the three problem solving tests 
are shown in Table 2. The mean ranges for the Grades Two and Three are 
higher than the mean ranges for the Grade One test; this is probably due to 
the inclusion of more two- and three-digit numbers on the tests and thus 
the greater opportunity for answers to vary widely. Paired two-tailed t- 
tests conducted on these data indicate that the range of answers narrowed 
significantly from the pretest to the posttest on the Grades One and Two 
tests (Gr. 1: t(24)=2.45, a < -05; Gr. 2: t(30)=2.25, £.< .05). There were no 
significant differences observed on the Grade Three test. Students in the 
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two younger grades seemed to be honing in on the right answer by 
posttest time, even if they did not get it correct. 

Insert Table 2 about here 



B. Types of Common Errors , The ideal situation for diagnosing 
student errors is to interview students at the time when they produce the 
errors - in the course of problem solving. Due to the scope and purpose of 
the assessment, we were not able to io this; only the errors were available 
for analysis. 

We examined the common errors for each item, and inferred what 
strategy students might have been using in order to arrive at these errors. 
These were grouped into five categories of common errors: 

1. Computational Errors were answers that were ±1 digit (and, in a 
few cases, ±2 digits) away from the correct answer. The problem 
solving strategy was probably correct, but the student miscounted 
and, thus, got an correct answer. 

2. Operational Errors indicated the use of an incorrect arithmetical 
operation (e.g., subtraction instead of addition, addition instead of 
multiplication, etc.). These kinds of errors probably indicate that 
the student either does not understand what the signs (+, x, ^) 
mean or does not understand why a particular operation is 
required. 

3. Conceptual/Procedural Errors include a broader range of errors. 
These errors included those where students appeared to be using 
an appropriate procedure incorrectly (e.g., "borrowing" from the 
tens column without subtracting a ten), which could be a good 
indication that the student making such a procedural error does 
not understand the conceptual rationale for the procedure. Also 
included in this category were those errors that dealt with 
misunderstandings of place value. 



Hojnacki & Grover 



21 



April 1992 



Thinking Mathematics and Students 



20 



4. Problem Misunderstanding Errors also include a broad range of 
errors. This category included all those errors which seemed to 
indicate that the student did not interpret the problem 
information correctly (e.g., only doing the first step of a two-step 
problem; not ignoring the extraneous information sometimes 
accompanying problems). The most frequent error of this type, 
which was especially common on the Grades One and Two tests, 
was to reiterate one of the quantities mentioned in the problem 
statement. 

5. Indeterminate Strategy Errors were errors for which no strategy 
could easily be inferred. This does not mean that students were 
just guessing, however. In fact, because at least three students 
had to produce a specific answer before it was classified as a 
common and not an idiosyncratic one, random guessing is even 
less likely. However, there was insufficient information to decide 
why students made these errors. 

Breakdowns of the percentages of common errors categorized into 
the five types for each grade's test are provided in Figure 5. The 
percentages of common errors out of total responses (including correct 
answers and idiosyncratic errors) for each grade's test was displayed in 
Table 1 and are summarized below. 



Insert Figure 5 about here 



The most salient difference between the pre- and post-tests is, of 
course, the increase in the percentage of correct answers, which we have 
already discussed. Since there were fewer errors produced on the posttest, 
there were corresponding decreases in the percentages of each type of 
common error. On the Grade One test, the five types of common errors 
represent fully 38% of all answers on the pretest, but only 8% on the 
posttest. On the Grade Two test, the decrease was from 26% to 12%; on the 
Grade Three test, the decrease was from 28% to 14%. 
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The single consistent change across all three tests is that the 
percentage of Indeterminate Strategy Errors decreased the most from pre- 
to post-test (decreases were 21%, 26%, and 10%, respectively, on the 
Grades One, Two, and Three tests). Computational Errors were more 
frequent, especially on the posttests, for the two younger grades. On the 
Grade Three test, the two main error types (pre and post) were Operational 
Errors and Conceptual/Procedural Errors. This corresponds to the absolute 
increase in number of operations, concepts, and procedures students are 
asked to understand by Grade Three. 

C. Instructional Implications of Problem Solving Test Resuhs . The 
individual teacher is the best qualified to determine what the instructional 
implications of these resuhs are for his/her group of students. However, in 
the spirit of the reform efforts that advocate that assessments should 
inform instructional practice, we shared some of our own ideas with our 
pilot site teachers in the report of the student assessment (Hojnacki & 
Grover, 1991). We encouraged the teachers to discuss the assessment as a 
whole and in particular the instructional implications of the problem 
solving test results at their Thinking Mathematics professional 
development sessions. 

First, the classification of common errors that emerged from these 
data could provide a framework from which teachers can develop 
questioning techniques and strategies to probe more fully the thinking and 
reasoning patterns behind students* solution processes. In particular, one 
of the most frequent error types on the Grades One and Two lests in the 
Problem Misunderstanding category was the reiteration of one of the 
quantities mentioned in the problem statement. Why do students do this? 
Are they focusing erroneously on some language cues? Are they just 
guessing? Why do they believe the correct solution would be provided in 
the problem? 

Secondly, the value of having students articulate their reasoning 
seems to be reinforced by these results. Since Problem Misunderstanding 
errors were most common on the Grades One and Two tests, there may be 
more need to spend time with language and talking about the meaning of 
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the problem in a variety of ways (which is in fact one of the research- 
based principles of TM). The same remedy might also lower the incidence 
of Operational and Conceptual/Procedural errors on the Grade Three test, 
by getting students into the habit of talking through problems and 
procedures and understanding what information is being required. Also, 
additional probing by teachers, in the form of requesting students to 
describe and justify their solutions in a class setting, might shed some light 
on the Indeterminate Strategies solutions. It is possible that the common 
decrease observed for all grades in the percentage of Indeterminate 
Strategies at the posttest might be partially attributable to the yearlong 
effects of emphasizing students* making sense of their answers. An 
additional classification of errors might emerge from a teacher's probing 
and suggest particular instructional strategies to address those errors in 
thinking. Or, it may be that the errors would fall into one of the already 
established categories and suggest a greater preponderance of 
misconceptions in a particular area than these data indicate. 

D. Summary of Problem Solving Test Results . For all grades, 
accuracy increased dramatically and the range of errors narrowed from 
pre to post test. Error analyses indicated that Computational errors 
were more frequent on the Grades One and Two tests than on the Grade 
Three test (pre and post), while Operational and Conceptual/Procedural 
errors were the most common type observed on the Grade Three test. 
The percentage of idiosyncratic errors decreased from pre to post test 
for all grade levels. Teacher knowledge of these error patterns can be 
useful in classroom diagnosis and instructional planning. 

Conclusions 

How have the analyses just described informed us about student 
learning outcomes in the Thinking Mathematics program? What evidence 
of student empowerment via teacher empowerment is there? The 
available data on teachers* experiences with Thinking Mathematics (Gill & 
Murakami, 1992; Billups & Rauth, 1992) supports the view that teachers 
reported being changed in empowering ways. The project *s teacher self- 
report data show that these teachers perceived empowering changes in 
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their students, as well. Performance on the Wood-Cobb tests indicated 
notable improvements in student problem solving ability. Positive student 
attitudes towards mathematics were revealed by the project's attitude 
survey; students' lower motivation scores, however, suggest that they are 
not yet ready to match their efforts with their generally positive regard 
for mathematics. TM students performed as well as or better than their 
non-TM peers on both the Computational and the Concepts and 
Applications subsections of the standardized tests. Although the results 
reported from the 1990-91 Thinking Mathematics student assessment are 
preliminary, there are multiple indications that student learning and 
attitudes were enhanced by their participation in the program. The 
collaborative model developed by the Thinking Mathematics project seems 
to be paying off for both teachers and students. 

Along the way, we have discovered many difficulties inherent in 
attempting to assess student change as a function of instructional change. 
From a research perspective, these difficulties were compounded because 
of the completely voluntary, non-threarening, and non-judgmental nature 
of the instructional change program. With Thinking Mathematics currently 
in its second year of implementation, another student assessment is 
underway. The lack of control over the extent of TM implementation is 
unavoidable. However, we will be in a better position to evaluate change 
in student attitudes towards mathematics by pre- and post-testing. Also, 
we will be able to compare problem solving ability over two years. The 
additional data collected should allow us to more fully address questions 
concerning student outcomes. 
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Table 1 

Mean Percentages of Responses per Category (Pretest and Posttest') on each 
Problem Solving Test 









Pretest 


Posttest 


Paired one- 
tailed t-test 
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Correct Answers 
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77 


11.37* 






Common Errors 
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Idiosyncratic Errors 
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Correct Answers 
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Idiosyncratic Errors 
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Correct Answers 


45 
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11.12* 






Common Errors 


28 


14 








Id io sy nc rat ic Er ro rs 


27 


14 





*£ < .0001 
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Table 2 

Mean Range of Answers per Item for Problem Solving Tests 









PRETEST 


POSTTEST 


Paired 
t-test 


(Srade 
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Test 
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Testa 


286.8 


127.3 


2.25* 


Grade 
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Testa 


282.4 


279.8 


0.97 



Notes, 

a Two outlier items with ranges of over 1000 were removed from the 
analysis. 

*E < .05 
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Computational Test Scores 




Grade 1 Grade 2 Grade 3 Grade 4 Grade 5 



Figure 1 . Mean median national percentile (MDNP) scores for Thinking 
Mathematics classes and non-Thinking Mathematics classes on the 
Computational subtests of standardized achievement tests. 



Concepts and Applications Test Scores 




Grade 1 Grade 2 Grade 3 Grade 4 Grade 5 



Figure 2 . Mean median national percentile (MDNP) scores for Thinking 
Mathematics classes and non-Thinking Mathematics classes on the 
Concepts and Applications subtests of standardized achievement tests. 



Student Attitudes 




Grade 2 Grade 3 Grade 4 ALL 

GRADES 



Figure 3 . Mean percentages of positive responses to items on the Thinking 
Mathematics "Students' Attitudes Toward Mathematics" Survey, by scale 
and student grade level. 




Fi gure 4 . Mean percentages of correct responses on the Grades One, Two, 
and Three Wood-Cobb Problem Solving Tests (pretests and posttests). 



Distributions of Common Errors 
on Pretests and Posttests 



Grade 1 Test: 

Pre 
Post 



i 


1 









I 



1 00% 



Grade 2 Test: 



Pre 



Post 








m 




1 1 i i 1 



100% 




Post 



J 



100% 



Kinds of Common Errors: 

■Computational 

° Conceptual/Procedural 
° Operational 

^Problem Misunderstanding 
^ Indeterminate Strategy 



Figure 5 . Percentages of different types of common errors on the Thinking 
Mathematics Wood-Cobb Problem Solving Tests by grade, on pretests and 
posttests. 
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Appendix A 

The AFT/LRDC Thinking Mathematics Project 

The Thinkmg Mathematics (TM) project, a 54 month project begun in 
1987, is a collaborative effort of the American Federation of Teachers 
(AFT) and the University of Pittsburgh's Learning Research & Development 
Center (LRDC), sponsored by a grant from the National Science Foundation. 
The collaboration's primary objective is to develop more effective ways of 
disseminating new knowledge about mathematics instruction and learning, 
based upon the clinical wisdom of [expert] classroom mathematics teachers 
and upon current research findings. ^ It was hoped that by involving 
teachers (along with researchers) in all phases of the interpretation of the 
research, the development of professional development materials, and the 
dissemination process, the project would contribute toward bridging the 
research-practice gap. 

After three years of intense teacher-researcher dialogue, informal 
classroom implementation and informal teacher-to-teacher dissemination, 
and an incredible amount of writing and rewriting, TM reached a critical 
phase. The first volume. Thinking Mathematics: Foundations (Bodenhausen 
et al., in press), which covered the subject matter topics of counting, 
addition, and subtraction, entered its first year of formal pilot testing in 
1990-91. Teachers and students at six sites across the country — San 
Francisco, CA, Albuquerque, NM, Anderson, IN, Gary, IN, Hammond, IN, and 
Dade County, FL — participated. Classes ranged from kindergarten to 7th 
grade, including some bilingual and special education classes. The second 
volume. Thinking Mathematics: VoL 2 Extensions (Gill & Grover, in press), 
covered the subject matter topics of multiplication and division. Pilot 
testing for this volume occurred during 1991-92 at five of the six sites that 
piloted the Volume 1 materials. 



^ These research findings for various topics in elementary arithmetic were summarized 
by prominent educational researchers for a specially commissioned volume. Analysis of 
arithmetic for mathematics teaching , edited by Leinhardt, Putnam, and Hattrup (1992). 
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Theoretical Foundation of Thinking Mathematics: Ten Principles 

The first volume synthesizes the research findings which suggest a 
new approach to teaching mathematics that promotes the revolutionary 
idea that all students can leam mathematics. These research ideas suggest 
that teachers should embed instruction in problem solving activities, 
encourage students to explore, explain and justify procedures for solving 
problems, and should encourage discussion among the students so that 
they share their different approaches to solutions and articulate their 
reasoning in language that is understandable by their fellow students. 
This is not to say that children should be allowed to explore without 
direction in the hope that they will discover the important mathematical 
ideas on their own. The program of instruction is orchestrated by the 
teacher in an overall plan that integrates ideas to achieve particular goals. 
These ideas are elaborated in the project's written materials (Bodenhausen 
et al., in press; Gill & Grover, in press). 

The following Ten Principles summarize the collaboration's collected 
wisdom on what characterizes good, research-based mathematics 
instruction: 

1. Build from students* intuitive knowledge 

2. Establish a strong number sense 

3. Base instruction on situational story problems 

4. Use manipulatives to represent problem situations 

5. Require students to describe and justify their mathematical 
thinking 

6. Accept multiple correct solutions 

7. Balance conceptual and procedural learning 

8. Use a variety of teaching strategies 

9. Use ongoing assessment to guide instruction 
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10. Adjust the curricular timeline 

Teacher-to-teacher collaboration and support were discovered to be 
increasingly important aspects of the TM dissemination process of subject 
matter grounded research (Grover, Gill, & Kaduce, 1991; Billups, 1992; Gill 
& Murakami, 1992). The AFT's ten-year-old Educational Research & 
Dissemination network had always utilized practitioners in both the 
development of research translations to be used in teacher training and in 
the conducting of national ER&D training. However, the extent to which the 
newly-trained teachers needed sustained support and follow-up training 
grew more apparent in TM's pilot year. The opportunity to discuss with 
other practitioners what they saw happening or not happening in their 
classrooms was essential. TM was unlike any of the other ER&D training 
modules in its focus on learning and instruction in a particular subject 
matter — mathematics. The earlier ER&D training focused on general 
instructional practice, applicable to any subject matter. For example, some 
of the topics disseminated via ER&D concern beginning-of-year classroom 
management, cooperative small groups, critical thinking, and teacher 
praise. 

As is customary with ER&D, teacher-trainers at the pilot sites 
participated in a week-long summer training workshop (led by the 
Thinking Mathematics teacher developers), and attended a three-day mid- 
year reunion. But recognizing the increased needs for additional assistance 
and teacher-to-teacher collaboration, several practices novel to ER&D were 
initiated during the 1990-91 pilot year of Thinking Mathematics. Two of 
the five teacher developers of Thinking Mathematics were released from 
teaching in order to provide full-time support to the pilot teachers and 
teacher-trainers across the country. Teachers at the pilot sites were 
provided with ten release days during the year: five days for initial 
training and one day a month for follow-up training and group meetings. 
Teacher-trainers were provided with an additional release day each month 
for preparation and administrative responsibilities. Finally, a Thinking 
Mathematics newsletter (**The Right Angle") was published, with the goals 
of keeping Thinking Mathematics teachers connected to each other and 
which allowed them to share classroom experiences. 
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During the 1990-91 TM pilot year, data were collected from the 
teachers about changes in their instructional practice and the impact their 
participation in the project had on their perceptions of themselves as 
teachers. Sources of data were structured interviews, written surveys, and 
video or audio tape recordings of discussion sessions held at their home 
schools with their colleagues. These data provided some insights into the 
sense of empowerment teachers enjoyed as a resuU of their participation 
in the collaborative project. 

In addition, data were collected to provide insights into the impact 
on students enrolled in classes of teachers who participate in such projects. 
Four sources provided data on students: teacher self-report questionnaires 
to obtain their perceptions of the impact on their students, a student 
attitude survey, a problem solving assessment, and standardized test 
scores. 
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